An Analysis of Feature Selection and Reward Function for Model-Based Reinforcement Learning

In this paper, we propose a series of correlation-based feature selection methods for dealing with high dimensionality in feature-rich environments for modelbased Reinforcement Learning (RL). Real world RL tasks usually involve highdimensional feature spaces where standard RL methods often perform badly. Our proposed approach adopts correlation among state features as a selection criterion. The effectiveness of the proposed methods are compared against previous methods referred as 10PreviousFS [2] using the data from an intelligent logic tutor called Deep Thought (DT) [1]. We evaluated the effectiveness of different feature selection methods by expected cumulative reward (ECR) [3], considering two types of reward: immediate and delayed. Our results show that our proposed methods significantly outperform previous feature selection methods with both types of rewards. Moreover, the “best” policy induced using immediate reward differs significantly from that induced from delayed reward.